1. Field of the Invention
The present invention relates to an image signal converting method and an image signal converting apparatus, and is applicable to an up-converter for converting a standard resolution signal (hereinafter referred to as a standard definition (SD) image signal) of NTSC or the like to a high-resolution signal (hereinafter referred to as a high definition (HD) image signal) of high-vision or the like, an interpolating device for image signal for converting a sub-sampled signal to an original image signal, and the like.
2. Description of the Related Art
Though various pieces of digital equipment are released, a signal converting apparatus for realizing signal conversion between pieces of equipment is necessary for connection between pieces of equipment with different signal formats. For example, to display SD image data on an HD monitor, an up-converter for converting SD image data to HD image data is necessary.
In the case of this type of the up-converter, HD image data is generated by applying the frequency interpolation processing to SD image data with an interpolation filter and thereby, performing pixel interpolation. The interpolation processing is described below by referring to an example of spatial arrangement of SD and HD pixels shown in FIG. 1. In this connection, in FIG. 1, a continuous line represents a first field and a dotted line represents a second field. In FIG. 1, the number of HD pixels is two times larger than the number of SD pixels in horizontal and vertical directions respectively. When noticing the SD pixel shown by a symbol ".circleincircle." in FIG. 1, four types of HD pixels mode1, mode2, mode3, and mode4 are present nearby the SD pixel. Therefore, the up-converter generates these four types of HD pixels mode1, mode2, mode3, and mode4 by means of frequency interpolation processing.
Some up-converters with a simple structure generate the above four types of the HD pixels mode1, mode2, mode3, and mode4 from field data of SD image data. The interpolation filter used for the above operation includes an in-space two-dimensional non-separable filter 1 shown in FIG. 2 and a horizontally/vertically separable filter 5 shown in FIG. 3.
The two-dimensional non-separable filter 1 generates four types of HD pixels mode1, mode2, mode3, and mode4 by independently interpolating these HD pixels with two-dimensional filters 2A to 2D and obtains HD image data by serializing the interpolation results in a selecting circuit 3. The horizontally/vertically separable interpolation filter 5 generates two scanning-line data values by performing processings for the HD pixels mode1 and mode3 with a vertical interpolation filter 6A and processings for the HD pixels mode2 and mode4 with a vertical interpolation filter 6B. Then, the filter 5 generates final four types of the HD pixels mode1, mode2, mode3, and mode4 by using horizontal interpolation filters 7A and 7B for each scanning line and obtains HD image data by serializing the HD pixels in a selecting circuit 8.
In the case of the above conventional up-converter, however, even if an ideal filter is used as the interpolation filter, the spatial resolution is the same as the case of an input SD image though the number of pixels increases so as to correspond to an HD format. Moreover, because the ideal filter cannot actually be used, there is the problem that it is impossible to generate an HD image with a resolution lower than that of an SD image.
To solve the above problem, an image signal converting apparatus and a method thereof applying the so-called classification adaptive processing method are proposed which obtains an HD pixel closer to a true value by classifying input SD image data values in accordance with a distribution of signal levels of the data values and performing predictive operation for each class with prediction coefficients previously obtained through learning. For example, such a method has been proposed in the specification and the drawings of Japanese Patent Application Laid-Open No. 5-328185, published on Dec. 10, 1993, corresponding to U.S. Pat. application Ser. No. 08/061,730, filed on May 17, 1993, by the present applicants.
An up-converter applying the classification adaptive processing is constituted as shown in FIG. 4. An up-converter 10 inputs input SD image data D1 to a classifying circuit 11 and a predictive operating circuit 12. The classifying circuit 11 sets a plurality of circumferential pixels about a noticed pixel to the input SD image data D1 (e.g. 8-bit pulse code modulation (PCM) data) as a classification pixel (hereinafter referred to as a classification tap) and generates a class data D2 in accordance with its waveform characteristic (level distribution pattern). In this connection, in FIGS. 5A and 5B, a continuous line represents a first field and a dotted line represents a second field.
In this case, to generate the class data D2 by the classifying circuit 11, the following methods are considered: a method of directly using PCM data (that is, a method of directly using PCM data as class data D2) and a method of decreasing the number of classes by using a compression method such as ADRC (Adaptive Dynamic Range Coding), DPCM (Differential Pulse Code Modulation), or VQ (Vector Quantization). In the above methods, the method of directly using PCM data as the class data D2 has the problem for practical use that the number of classes becomes a very large value of 2.sup.56 because 8-bit data equivalent to seven pixels is present when using, for example, classification taps consisting of seven pixels as shown in FIG. 5A.
Therefore, actually, the number of classes is decreased by a compression method such as ADRC. For example, when one-bit ADRC for compressing each pixel to one bit is applied to seven pixels set as classification taps, the number of classes can be decreased to 128 because the minimum value of seven pixels is removed in accordance with a dynamic range defined from seven-pixel data and then the pixel value of each tap is adaptively quantized by one bit. ADRC is developed as a signal compression method for an image signal, which is suitable to express the waveform characteristic of an input signal with a small number of classes.
The prediction coefficient data D3 previously obtained through learning is output from the prediction coefficient read only memory (ROM) 13 by using the class data D2 as a read address and supplied to the predictive operating circuit 12. The predictive operating circuit 12, as shown in FIG. 5B, sets a plurality of circumferential pixels about a noticed pixel as a predictive-operation pixel (hereinafter referred to as a prediction tap), and estimates and outputs HD image data D4 by using each pixel value constituting the prediction tap and the prediction coefficient data D3 and thereby, performing product-sum operation shown by the following equation (1). This HD image data D4 is supplied to a converting circuit 14 to be converted to time-sequential HD image data D5, and this is displayed on a screen which is not shown. ##EQU1##
In the equation (1), it is assumed that estimated HD pixel value is y', each prediction tap pixel value is x.sub.i, and prediction coefficient is w.sub.i. Moreover, in this case, "n" in the equation (1) comes to 13 because the number of pixels forming the prediction taps is 13 as shown in FIG. 5B.
In this case, the prediction coefficient data D3 for each class stored in the prediction coefficient ROM 13 is previously obtained through the learning using HD image data. The learning procedure is described below by referring to FIG. 6. After the learning procedure is started in step SP1, learning data is first generated by using an already-known HD image in order to learn a prediction coefficient in step SP2.
In step SP3, it is decided whether learning data necessary enough to obtain a prediction coefficient is collected. When it is decided that learning data is necessary, it proceeds to step SP4. When it is decided that sufficient learning data is obtained, it proceeds to step SP6. In step SP4, learning data is classified. This classification is performed in accordance with the processing same as that performed in the classifying circuit 11 (FIG. 4) of the up-converter 10 described the above. In this case, influence of noises is generally eliminated by excluding objects with a small data-change activity from learning objects.
Then, in step SP5, a normal equation is set up for each class in accordance with classified learning data. The processing in step SP6 is specifically described below. In this case, a case is described for generalization in which one pixel (HD pixel) to be interpolated is expressed by n SD pixels. First, the following equation (2) is set up by expressing the relation between each of pixel levels x.sub.1, . . . , x.sub.n of SD image data and the pixel level y of a noticed interpolation pixel in the form of a linear primary coupling equation of n taps according to prediction coefficients w.sub.1, . . . , w.sub.n for each class.
It is necessary to obtain the prediction coefficients w.sub.1, . . . , w.sub.n in the equation (2). ##EQU2##
A solution according to the method of least squares is considered to obtain the prediction coefficients w.sub.1, . . . , w.sub.n. In the case of this solution, it is assumed that X is SD pixel data, W is a prediction coefficient, and Y is noticed interpolation pixel data to collect data so as to set up an observation equation of the following equation (3). EQU XW=Y
where ##EQU3##
In the equation (3), m represents the number of learning data values and n represents the number of prediction taps (that is, n=13) to be set by the predictive operating circuit 12 (FIG. 4).
The following residual equation (4) is set up in accordance with the observation equation of the equation (3). EQU XW=Y+E
where ##EQU4##
From the equation (4), it is considered that the most probable value of each w.sub.i is obtained when a condition for minimizing the following equation (5) is effected. ##EQU5##
That is, it is necessary to consider the condition of the following equation (6). ##EQU6##
It is necessary to consider n conditions according to i of the equation (6) and calculate the prediction coefficients w.sub.1, . . . , w.sub.n to meet the n conditions. Therefore, the following equation (7) is obtained from the residual equation (4). ##EQU7##
Moreover, the following equation (8) is obtained from the equations (6) and (7). ##EQU8##
Then, the normal equation of the following equation (9) can be obtained from the equations (4) and (8). ##EQU9##
Because the normal equation of the equation (8) is n simultaneous equations having n unknowns, it is possible to obtain the most probable value of each w.sub.i from the n simultaneous equations. Actually, the simultaneous equations are solved by the sweeping-out method (elimination method of Gauss-Jordan).
In the case of the prediction learning procedure in FIG. 6, a loop of steps SP2-SP3-SP4-SP5-SP2 is repeated until normal equations equal to the number of unknowns n are set up in order to obtain undetermined coefficients w.sub.1, . . . , w.sub.n for each class. When a necessary number of normal equations are obtained in the above manner, an affirmative result is obtained in step SP3 and prediction-coefficient determination is started in step SP6.
In step SP6, the normal equation of the equation (9) is solved to determine the prediction coefficients w.sub.1, . . . , w.sub.n for each class. The prediction coefficients thus obtained are entered in storage means such as a ROM (that is, the prediction coefficient ROM 13 (FIG. 4)) which is address-divided for each class in the next step SP7. Prediction coefficient data for classification adaptive processing is generated by the above learning and the prediction learning procedure is terminated in the next step SP8.
A learning circuit 20 shown in FIG. 7 is considered as hardware structure for realizing the prediction learning processing. The learning circuit 20 converts HD image data to SD image data through a vertical thinning filter 21 and a horizontal thinning filter 22 and supplies the SD image data to a classifying circuit 23. In this case, the classifying circuit 23 has the same structure as the classifying circuit 11 of the above-described up-converter 10 (FIG. 4), sets a classification tap from the SD image data, and generates class data D2' in accordance with the waveform characteristic of the tap. The classifying circuit 23 is a circuit for executing the class determination processing (step SP4) in FIG. 6. The classifying circuit 23 transmits the generated class data D2' to a coefficient selecting circuit 24.
The coefficient selecting circuit 24 is a circuit for setting up a normal equation and moreover determining a prediction coefficient. That is, the coefficient selecting circuit 24 sets up the normal equation of the equation (9) for each class shown by the class data D2 by using the SD image data and HD image data and obtains a prediction coefficient from the normal equation. Then, the circuit 24 stores the obtained prediction coefficient in a corresponding class address of the prediction coefficient ROM 13.
However, to generate HD image data by the classification adaptive processing, there is a problem that the prediction accuracy by an up-converter is deteriorated unless a proper classification processing is performed in accordance with the feature of input SD image data when generating a prediction coefficient through learning. That is, to predict HD image data closer to a true value, it is important to collect only SD image data with a similar characteristic to generate each class and learn HD image data corresponding to each class as a teacher signal.
However, when the classification capacity is insufficient, HD image data which should originally be classified into different classes is classified into the same class. Therefore, a prediction coefficient obtained through learning is obtained from an average value of HD image data with different properties. As a result, an up-converter for predicting an HD image signal by using the prediction coefficient has a problem that a resolution creating capacity is deteriorated.